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Real Time PCR platform continued 


Executive Summary 


Currently, reverse transcription polymerase chain reaction (RT-PCR) performed on upper respiratory tract 
swab samples is the main approach used in Australian laboratories to detect of SARS-CoV-2, the virus that 
causes COVID-19 disease’. To meet the urgent need for diagnostic testing in Australia the Therapeutic Goods 
Administration (TGA) has expedited approvals (with conditions) for SARS-CoV-2 diagnostic tests based on the 
studies that manufacturers were able to provide at the time of application. To supplement this data, robust 
post market validation of SARS-CoV-2 diagnostic kits that are listed on the Australian Register of Therapeutic 


Goods (ARTG) is desirable. 


This study presents a post-market validation study of the Beijing Genomics institute (BGI) MGISP-960 robotic 
nucleic extraction instrument, paired with either the SLAN 96S real-time PCR analyser, or Bioer Line Gene 
9600 PCR analyser, and using the MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit (Ref 1000021043), 
and BGI Real-time fluorescent RT-PCR kit for detecting 2019-nCoV (Ref MFGO30010) collectively referred to 
here as ‘the BGI-Platform’. This equipment and reagents have been made available to selected Australian 


laboratories for diagnostic use. 


The study employed a 271 sample validation panel comprising 156 positive and 115 negative samples, 
randomly assembled. These were of necessity simulated samples, using inactivated cell-culture-grown SARS- 
CoV-2, and other respiratory viruses spiked into a representative variety of fluids, since patient samples were 


not available in sufficient volumes to parallel test in three laboratories. 
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Real Time PCR platform continued 


The BGI equipment and test kits were tested at two collaborating laboratories: ‘Laboratory 2’, and 
‘Laboratory3’. Both laboratories tested the whole 271 sample panel in parallel with the Victorian Infectious 


Diseases Reference Laboratory, Doherty Institute. 


A set of exercises only done in Melbourne separately compared the BGI MGISP-960 robotic nucleic extraction 
instrument, and the SLAN 96S real-time PCR analyser with Doherty Institute test methods. In addition 
different lot numbers of MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit (Ref 1000021043), and BGI 
Real-time fluorescent RT-PCR kit for detecting 2019-nCoV (Ref MFG030010) were compared with each other 


and Doherty Institute reagents. 


The most dilute sample reliably detected (Limit of detection, LOD) was calculated for each test, along with 
the accuracy in detecting true positive samples and true negative samples (Operational Sensitivity and 
specificity), the latter including samples containing no target, and samples containing respiratory viruses 
other than SARS-CoV-2. Precision in testing replicates of the same sample was examined compared to an 
objective standard test (droplet digital PCR), together with comparison of the graphical distribution of raw 


RT-PCR test output values (Cycle threshold values, Ct values) for each laboratory. 


In summary, the BGI platform was shown to have a Limit of Detection, and Operational Sensitivity and 
Specificity that were comparable to the reference test methods; Sensitivity being slightly higher and 
specificity slightly lower. This was fairly consistently achieved in two BGl-user Laboratories. The 


manufacturer’s Instructions for Use (IFU) dictate retesting of samples with low reactivity (Ct values above 
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Real Time PCR platform continued 


38), and a number of such values were recorded on testing negative panel samples, especially by one 
laboratory. Best practice would be to use a second assay using a different gene target for this retesting, 
consistent with Australian guidelines. This extra testing may be onerous for test users if low reactivity is 


commonplace in clinical use of the test, as this study suggests might be the case in some scenarios. 


One lot number of BGI Real-time fluorescent RT-PCR reagents gave noticeably lower specificity than another, 
albeit still relatively high, and varied considerably from the overall specificity achieved testing the panel that 
is described above. Otherwise different lot numbers of MGIEasy Magnetic Beads Virus DNA/RNA Extraction 
Kit and BGI Real-time fluorescent RT-PCR reagents gave comparable performance. Some differences in 
precision between replicates of the same sample were observed, but these were most evident in one sample 
type: (swabs diluted in UTM) and with one batch of MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit 


reagents. Overall the precision was acceptable for a qualitative test. 


Hence, no single issue has been identified to suggest that the BGI platform is not fit for purpose in Australian 
laboratories. Best practice in executing confirmatory testing of samples producing Ct values > 38 will be 
important in use. Similarly, close monitoring of consistency in performance of Lot numbers, especially of RT- 


PCR reagents, will be important to ensure optimal performance in clinical use 
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1. Introduction 


Timely, scalable and accurate diagnostic testing for SARS-CoV-2 underpins the public health response to 
COVID-19 disease, and clinical care of the infected individual. RT-PCR performed on respiratory samples is 
the main diagnostic modality for detection of acute COVID-19 disease." This is being done in Australia under 
an emergency exemption from the Therapeutic Goods Administration (TGA), initially for Public Health 
Laboratory Network of Australasia (PHLN)-member public health laboratories, and more recently extended 


to hospital and community laboratories. 


To meet the urgent need for diagnostic testing in Australia the TGA has expedited approvals (with conditions) 
for SARS-CoV-2 diagnostic tests based on the studies that manufacturers were able to provide at the time of 
application. To supplement this data, robust post market validation of SARS-CoV-2 diagnostic kits that are 


listed on the Australian Register of Therapeutic Goods (ARTG) is desirable. 


Here, we present a post-market validation study of the Beijing Genomics institute (BGI) MGISP-960 robotic 
nucleic extraction instrument, together with either the SLAN 96S real-time PCR analyser, or Bioer LineGene 
9600 PCR analyser, and using the MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit (Ref 1000021043), 
and BGI Real-time fluorescent RT-PCR kit for detecting 2019-nCoV (Ref MFGO30010). This equipment and 


reagents have been made available to selected Australian laboratories for diagnostic use. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


2. Methods 


2.1 Participating Laboratories 


The three participating laboratories in this study were Laboratory 1: Victorian Infectious Diseases Reference 


Laboratory (VIDRL), Doherty Institute, Melbourne, ‘Laboratory 2’, and ‘Laboratory3’. 


2.2 Validation Panel 


A sample panel was developed to test performance of the BGI CoVID-19 RT-PCR Platforms. The panel 
comprises 271 mock respiratory tract samples. There were 156 positive and 115 negative samples. Samples 
were randomised within the validation panel as presented to testing laboratories. 

Positive samples were all dilutions of gamma-irradiated (inactivated) SARS-CoV-2 (strain VIC/01/202). 
Samples were prepared by spiking a dilution series of gamma-irradiated SARS-CoV-2 into each sample matrix 
Actual residual clinical samples were not available in sufficient volumes to permit parallel testing in three 
laboratories. In addition replicating real clinical samples obviates potential uncertainty about samples 
discrepant between reference and test systems, since known quantities of virus are present in each positive 


sample. The preparation of virus and spiked samples is described in greater detail below. 
The positive samples comprise two groups representing samples for which the MGISP-960/MGIEasy 


Magnetic Beads Virus DNA/RNA Extraction Kit is validated according to the manufacturer’s Instructions for 


Use (IFU)?. The first is a four-fold dilution series of gamma-irradiated SARS-CoV-2 virus isolate in virus 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


transport medium (VTM) to simulate eluate from respiratory swabs. These were diluted 1/4000 to 
1/262,144,000 with quadruplicate replicates of each (a total of 36 samples). The second comprise VTM (n= 
72), 10% saliva diluted in VTM (n=24), and de-identified previously negative human nose and throat swabs 
in commercial transport medium (Copan UTM, CA USA)) (n=24), spiked with differing final dilutions of 


gamma-irradiated SARS-CoV-2 virus stock. 


The remaining samples were negative controls (n=115) which were a mixture of previously screened negative 
human swabs (n=33), 10% saliva diluted in VTM (n=49), VTM (n=9), and VTM or 10% saliva diluted in VTM 
spiked with other seasonal respiratory viruses including influenza A, influenza B, Parainfluenza 1,2 and 3, 
rhinovirus, enterovirus (Coxsackie A16), respiratory syncytial virus (RSV), human metapneumovirus, 
adenovirus, measles virus, parvovirus and seasonal human coronaviruses 229E and OC43 (n=24). These were 
a dilution of cell culture grown isolates from the Doherty Institute’s reference collection with Ct values 


ranging from 25 to 30 (data not shown). 

This complete panel of 271 samples was used to compare the performance of the complete nucleic acid 
extraction and RT-PCR workflows of the three participating laboratories. Sensitivity and specificity were 
compared, and LOD was examined using the four fold dilution series present in four fold replicates that is 


described above. 


A full listing of panel samples is provided in Appendix A. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


A subset of samples n=90, comprising two duplicate sets of 45 samples, each comprising 15 positive samples, 
and 30 negative samples, was used to perform a sub-study which individually compared the MGISP-960 high- 
throughput automated extraction and liquid handling workstation (MGI Tech Co., Ltd., China), and the BGI 
Real-time fluorescent RT-PCR kit for detecting SARS-CoV-2 on the BGI SLAN-96S real-time PCR system (BGI, 
China) with the respective reference methods. In addition this sub-panel was used to compare performance 
of different lot numbers of the MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit (MGI Tech Co., Ltd., 
China), and the BGI Real-time fluorescent RT-PCR kit for detecting SARS-CoV-2, with each other. 


Virus 

Virus stock for artificial spiking of samples was SARS-CoV-2/Vic/01/2020 isolate grown in vero cells that had 
been subjected to 50Kgray of gamma-irradiation and verified as inactive by subsequent virus isolation 
attempts and sub-culture passaging. Primary stocks of 1/100 and 1/1000 dilution of virus were prepared in 


VTM for subsequent dilution into all matrices used for preparation of mock-positive samples. 


Mock negative and positive samples 

Samples were prepared using 3 alternative dilution matrices. These include VTM prepared in-house 
(Essential Minimum Eagles medium (EMEM) (Sigma, UK) with 2% Foetal Bovine serum (FBS)(Bovogen Keilor 
East, Australia); VTM with 10% volume of cell-clarified human saliva) and SARS-CoV-2 negative screened 
human nose and throat swabs in Copan swabs in Universal Transport Medium (UTM) (Copan 306C, CA USA). 


Mock positive samples had varying dilutions of stock gamma-irradiated virus added to each sample pool prior 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


to aliquoting into individual replicates. Mock negative samples had no addition of gamma-irradiated virus 


added. 


After preparation and assembly of the test panel samples, aliquoting of 3 identical copies in 1.5ml tubes 
(Cat # 2231-50, SSI bio), numbered 1-18 to 101-353 was done at the Doherty Institute. All three panels were 


then frozen at -80°C. 


One test panel was provided to Laboratory 3 and then thawed for testing. A second panel was thawed for 
testing at Laboratory 1. The third panel was shipped frozen to Laboratory 2. The panel was verified received 


in good condition, and was subsequently thawed for testing. 


2.3 Test Methods 


2.3.1 Doherty Institute 


2.3.1.1 Nucleic Acid Extraction and Reverse transcription 


The validation panel samples for SARS-CoV-2 RNA detection were extracted using a QlAamp 96 DNA QlAcube 
HT kit (Ref 51331, Lot No. 166014792 Qiagen, Germany) on the QlAcube HT System (Qiagen, Germany) 
according to manufacturer’s instructions. In brief, a 200uL volume of each sample was used for extraction 
and eluted in a 60uL volume of elution buffer into a 96-well elution tray. Reverse transcription of viral RNA 
to complimentary DNA (cDNA) was performed using Bioline SensiFAST cDNA synthesis kit (Cat. No. CSA- 
01148, Lot BO81600 Bioline Reagents Ltd. UK) according to manufacturers instruction. In brief, a 10uL 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


volume of eluted nucleic acid was added to 10uL of SensiFAST cDNA reaction mix, heat linearized at 25°C for 
10 minutes then incubated at 42°C for 15 minutes before denaturation at 85 °C for 5 minutes on an Applied 


Biosystems SimpliAmp thermal cycler (Thermo Fisher Scientific, Singapore). 


2.3.1.2 SARS-CoV-2 real-time PCR 


An in-house RT-PCR TaqMan assay targeting the SARS-CoV-2 RdRP gene was used. Primers and probe 
sequences targeting the RdRP gene have been previously described by Caly et al. Briefly, 3ul of cDNA was 
added to a commercial real-time PCR master mix (Ref. z-PFAST-LR-10ml, Lot JN-00037-0007 PrecisionFast 
qPCR Master Mix with LowRox, Primer Design Ltd.,UK) in a 20ul reaction mix containing primers and probe 
with a final concentration of 0.9uM and 0.2uM for primer and the probe, respectively. An in-house positive 
extraction control, a negative control and a positive control were included with each PCR run. Thermal 
cycling and real-time PCR analysis for all assays were performed on an ABI 7500 FAST real-time PCR system 
(Applied Biosystems, Foster City, CA) with the thermal cycling profile: 95°C for 2 min, followed by 45 PCR 
cycles of 95°C for 5 sec and 60°C for 25 sec. Results were interpreted individually per run of 96 samples. A 
positive result was interpreted from uniform S-shaped amplification profiles where the threshold line was 
set to intersect with the mid-point of the linear amplification phase of the amplification-curve. Any result up 


to Ct of 40 is considered positive. Cycle thresholds of 40-45 are examined with further testing. 


2.3.2 Laboratory 2 
2.3.2.1 Nucleic Acid Extraction 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


The validation panel samples for SARS-CoV-2 were extracted using a MGIEasy Magnetic Beads Virus 
DNA/RNA Extraction Kit (MGI Tech Co., Ltd., China) (Ref 1000021043, Lot No. H0056) on the MGISP-960 high- 
throughput automated extraction and liquid handling workstation (MGI Tech Co., Ltd., China) according to 
manufacturer’s Instructions For Use (IFU) 3. In brief, 180uL of each sample was used for extraction and eluted 


in 30uL of elution buffer into a 96-well elution tray. 


2.3.2.2 SARS-CoV-2 real-time PCR 


A BGI Real-time fluorescent RT-PCR kit for detecting SARS-CoV-2 was used (Lot 6220200425) according to 
the manufacturer’s IFU*. This assay uses primers and probe sequences targeting the ORFlab gene. An 
internal control (Human B-actin), a negative control and a positive control were included with each PCR run. 
10ul of nucleic acid extract was added to the PCR mix. Thermal cycling and real-time PCR analysis were 
performed on the Bioer LineGene 9600 real-time PCR detection system (Bioer Technology, China) with the 
thermal cycling profile: 50°C for 20 min, 95°C for 10 min, followed by 40 PCR cycles of 95°C for 15 sec and 
60°C for 30 sec. Results were interpreted individually per run of 96 samples. A positive result was interpreted 
from uniform S-shape amplification profiles with the Ct value <38. A negative result is required to have a 
satisfactory Human B-actin Ct of <32. Results with unsatisfactory Human B-actin amplification are to be re- 
tested unless a uniform S-shape amplification profile of the SARS-CoV-2 target with the Ct value <38 is 


present. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
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2.3.3 Laboratory 3 
2.3.3.1 Nucleic Acid Extraction 


The Validation panel samples for SARS-CoV-2 were extracted using a MGIEasy Magnetic Beads Virus 
DNA/RNA Extraction Kit (MGI Tech Co., Ltd., China). Two extraction kit lot numbers (Ref 1000021043, Lot 
No.s H0058 and H0059) were used for 90 samples from the panel that had duplicate samples, so that 45 
samples were extracted using both lot numbers. The remaining validation panel was extracted using kit lot 
No H0059. Extraction was performed on the MGISP-960 high-throughput automated extraction and liquid 


handling workstation (MGI Tech Co., Ltd., China) according to manufacturer’s IFU?. 


2.3.3.2 SARS-CoV-2 real-time PCR 
The above mentioned BGI Real-time fluorescent RT-PCR kit for detecting SARS-CoV-2 Lot # 6220200318 and 
#62220200319 was used according to the manufacturer’s IFU*. An additional negative (PBS) and positive (y 


-irradiated SARS-CoV-2 VIDRL isolate diluted 1/1,000) extraction controls were also included with each PCR 
run. A 10ul volume of nucleic acid extract was added to the PCR mix. Thermal cycling and real-time PCR 
analysis were performed on the BGI SLAN-96S real-time PCR system (BGI, China) with the above mentioned 
thermal cycling profile. Results were interpreted individually per run of 96 samples. Results were interpreted 
individually per run of 96 samples. A positive result was interpreted from uniform S-shape amplification 
profiles with the Ct value 238. A negative result is required to have a satisfactory Human B-actin Ct of < 32. 
Results with unsatisfactory Human B-actin amplification are to be re-tested unless a uniform S-shape 


amplification profile of the SARS-CoV-2 target with the Ct value < 38 is present. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


2.3.4 Digital Droplet PCR for Absolute Quantification of SARS-CoV-2, Doherty Institute 


A 4-fold dilution series (1/4000 to 1/262,144,000) of spiked gamma irradiated SARS-CoV-2 was prepared in 
3 different sample matrices as described above (i.e. VTM, 10% saliva diluted in VTM and negative human 
nose and throat swabs in commercial transport media). For each dilution point two separate extractions was 
performed using the QlAamp 96 DNA QlAcube HT kit on the QlAcube HT system. The purified nucleic acid 
was then converted to cDNA as described above. Prior to performing digital droplet PCR (ddPCR) the cDNA 
samples were first tested in an in-house qualitative assay, using targets RdRP gene? and N genet, to 
determine the sample’s relative viral load through the cycle threshold (Ct) measure for each gene target. PCR 
amplification was carried out with the same thermal cycling profile as the in-house RdRP RT-PCR assay. 

For ddPCR, primers and probe from the qualitative N gene RT-PCR was used to prepare a ddPCR Supermix 
(ddPCR Supermix for Probes, Cat. #1863024, Batch 64330928, Bio-Rad, Bio-Rad Laboratories. Inc, USA). The 
Supermix was prepared with the same concentration of primers and probe as the qualitative assay. 3ul of 
cDNA template was added to the Supermix for a final reaction volume of 25ul. Droplets were generated from 
this mixture on the QX200 Droplet Generator (Bio-Rad, USA) as per manufacturer’s instructions and run on 
a thermal cycler (Thermo Scientific Arktik Thermal Cycler, Thermo Fisher Scientific) with thermal cycling 
conditions: 95°C for 10 min, followed by 40 cycles of 95°C for 30 sec and 60°C for 1 min, 98°C for 10 min and 
hold 4°C. A QX200 Droplet Reader (Bio-Rad, USA) was used to take fluorescence readings. Each dilution point 
was tested in the ddPCR in quadruplicate (2 replicates for each of the 2 extracts per dilution point) and the 


mean quantity was calculated. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


2.3.5 Statistical analysis 


Data analysis was managed using GraphPad Prism (v7.05). Nonlinear regression, Gaussian distribution and 


One-Way ANOVA with Dunn's multiple comparisons, were performed as deployed in GraphPad Prism. 


2.4 Validation Method 

2.4.1 On-Site Validation Laboratory 3 

Performance of the MGlEasy Magnetic Beads Virus DNA/RNA Extraction Kit on the MGISP-960 high- 
throughput automated extraction workstation, and BGI Real-time fluorescent RT-PCR SLAN 96-S RT-PCR 
machine (collectively referred to below as the ‘BGI Platform’) were compared head to head with the Doherty 
Institute QIAGEN QlAcube extraction robot and Applied Biosystems 7500 RT-PCR machine using Doherty 
Institute in-house RdRP gene RT-PCR. 


The BGI platform and test were examined in situ at Laboratory 3. Protocols used for the BGI platform were 
the Laboratory 3 standard operating procedures (SOP) for this equipment, and assays were performed 


according to the manufacturer’s IFU? 4. 


Personnel were an experienced scientist from Doherty Institute’s Virus Identification Laboratory, working 


collaboratively with an experienced Laboratory 3 Molecular scientist. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


In the first exercise (Fig 1. Exercise 1) the whole BGI platform comprising MGISP-960 Extraction Robot, and 
SLAN 96-S RT-PCR machine with the Doherty Institute reference platform, in testing the entire 271 sample 


panel. 


Figure 1. Exercise #1 


(1) Doherty Institute 


© QIAGEN QlAcube Applied Biosystems 7500 


(2) Laboratory 2 





271 
Samples 


BGI MG ISP-960 Bioer LineGene PCR 
Extraction Robot System 
Lot # H0056 Lot # 6220200425 


(3) Laboratory 3 


271 BGI MG ISP-960 BGI SLAN 96S RT-PCR 
Samples Extraction Robot Lot # 6220200318 & 
Lot # H0058 & # H0059 # 6220200319 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


2.4.2 Validation Panel Analysis Laboratory 2 


The Validation panel described above was also shipped to Laboratory 2 as described above. Testing of the 
271 sample panel (Fig 1. Exercise 1) at Laboratory 2 compared the performance of the BGI MGISP-960 
extraction robot, and Bioer LineGene PCR System RT-PCR machine, with MGIEasy Magnetic Beads Virus 
DNA/RNA Extraction Kit and BGI Real-time fluorescent RT-PCR compared head to head with the Doherty 


Institute Reference platform and reagents. 


Protocols were the Laboratory 2 Standard Operating Procedures (SOP), and assays were performed 


according to the manufacturer’s IFU2*. Personnel were Laboratory 2 Senior Molecular Scientific Staff. 


2.4.3 Sub-study Laboratory 3 


An additional sub-study was done as part of the Laboratory 3 on-site evaluation using 90 selected samples 
from within the 271 sample panel. The 90 samples comprised two duplicate sets of 45 samples, each made 
up of 15 positive and 30 negative samples, as described above in Section 2.1. These samples were used to 


make the following comparisons: 
Firstly (Fig 2. Laboratory 3 Sub-study) RT-PCR was compared between the BGI Real-time fluorescent RT- 


PCR kit using the SLAN 96-S RT-PCR analyser, and the Doherty Institute ABI 7500 RdRP in-house RT-PCR 


assay. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


Secondly (Fig 2. Laboratory 3 Sub-study) nucleic acid extraction using the BGI MGISP-960 high-throughput 


automated extraction workstation was compared to nucleic acid extraction using the QIAGEN QlAcube. 


Thirdly (Fig 2. Laboratory 3 Sub-study) performance was compared for two different lot numbers of the 
MGlEasy Magnetic Beads Virus DNA/RNA Extraction Kit (#H0058, and #HO059), and two different lot 
numbers of the BGI Real-time fluorescent RT-PCR kit for detecting SARS-CoV-2 (#6220200318, and 
#6220200319). 


To execute these studies separate copies of the 90 sample panel underwent nucleic acid extraction at the 
Doherty Institute using the QIAGEN QlAcube, and at Laboratory 3 using the BGI MGISP-960 high-throughput 
automated extraction workstation. The resulting eluates were split into two aliquots and frozen at -80C. Each 
aliquot was subsequently tested, one at Laboratory 3 using the BGI Real-time fluorescent RT-PCR kit and 
SLAN 96-S RT-PCR machine protocol, and the other underwent RT-PCR analysis using the Doherty Institute 
ABI 7500 RdRP in-house RT-PCR assay. 


At Laboratory 3 45 samples were extracted using MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit Lot 
#HO058, and the other 45 duplicate samples using Lot #HOO59. In turn 45 extraction eluates underwent RT- 
PCR at Laboratory 3 using the BGI Real-time fluorescent RT-PCR kit for detecting SARS-CoV-2 Lot 
#6220200318, and 45 duplicate extraction eluates underwent RT-PCR using Lot #6220200319. Equipment, 


SOP, assay performance, and personnel were as described above. 
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Figure 2. Exercise #2 


Laboratory 3: Sub-study 











Applied Biosystems 
7500 
BGI MG ISP-960 

BGI SLAN 96S RT-PCR 
MPS-PCR Lot # 6220200318 

BGI SLAN 96S RT-PCR 
MPS-PCR Lot # 6220200318 

BGI SLAN 96S RT-PCR 
MPS-PCR Lot # 6220200319 


QIAGEN QlAcube BGI SLAN 96S RT-PCR 


MPS-PCR Lot # 6220200318 


Applied Biosystems 
7500 


The sub-analysis described above in which the performance of BGI extraction robot, and BGI RT-PCR analyser 





Extraction Robot 
Lot # H0058 


oJ) 


BGI MG ISP-960 
Samples 


Extraction Robot 
Lot # H0059 





were separately examined at Laboratory 3 was not attempted with Laboratory 2. It was considered not 
feasible to ship interstate the extracted eluates from the nucleic acid extraction robots of the Doherty 
Institute and Laboratory 2 due to the fragility of extracted viral RNA. The robustness of results of any RT-PCR 


analysis of RNA extracts after such shipping could not be relied on. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


3. Results 


3.1 Limits-of-Detection 
A 4-fold dilution series of virus in VTM starting at 1/4000 dilution of stock virus, as described above was used 


to determine LoD for each of the three laboratories. 


The LoD for each laboratory’s testing platform was defined as the highest dilution at which all four replicate 
samples were detected. Laboratory 1, Laboratory 2 and Laboratory 3 had LoD of 1/16,384,000, 1/65,536,000 
and 1/65,536,000, respectively (Table 1). It is worth noting that only 3 out of 4 replicates were detected at 
1/16,384,000 for Laboratory 2 but since all replicates were detected at the next dilution the latter was scored 


as the LoD. 
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Table 1. Limit-of-detection of SARS-CoV-2 for each laboratory testing platform was determined by testing 


serial dilutions of a stock virus. 


Laboratory 1? Laboratory 2° Laboratory 3° 


Sample Detections/ Ct value Detections/ Ct value Detections/ Ct value 


dilution? replicates Mean? (SD?) f replicates Mean? (SD?) | replicates (Mean?/SD?) 


1/4,000 25.0 (0.9) 4/4 21.8 (0.1) 21.4 (0.1) 
1/16,000 28.4 (0.9) 4/4 24.5 (0.1) 23.8 (0.3) 
1/64,000 29.7 (0.8) 4/4 26.5 (0.2) 25.8 (0.4) 
1/256,000 32.2 (0.4) 4/4 28.5 (0.1) 27.9 (0.3) 
1/1,024,000 33.8 (0.3) 4/4 30.7 (0.4) 29.9 (0.4) 


1/4,096,000 36.4 (0.4) 4/4 32.7 (0.1) 32.2 (0.5) 


1/16,384,000 37.8 (1.1) 3/4 34.3" (0.7) 34.3 (0.4) 


1/65,536,000 N/A 4/4 36.5 (0.2) 35.6 (0.7) 


1/262,144,000 40.2° 3/4 38.0“ (0.7°) 36.7* (0.7°) 





a Doherty Institute. > Laboratory 2. € Laboratory 3. 1 Diluted samples were from the VTM matrix. 2 Mean Ct is based on Ct values of 4 replicate 
samples. SD? Standard deviation. * Ct value based on a single detection. * Ct value based on the mean of 3 detected replicates. ê standard 
deviation from Ct values of 3 detections. N/A Not applicable. Grey shaded boxes indicate the LOD dilution point for each laboratory. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


3.2 Operational Sensitivity 
Sensitivity of the testing platform used in each laboratory was assessed on 271 samples. These were made 
up of 156 spiked positive samples and 115 true negative samples. The test sensitivity of Laboratory 1, 2 and 


3 was demonstrated to be 94.2%, 98.1% and 99.4%, respectively (Table 2). 


Table 2. Detection of positive SARS-CoV-2 spiked material (n=156) 


Laboratory 1° Laboratory 2° Laboratory 3° 


onse | anse 1/6 
cam | osm | sax 


a Doherty Institute. ® Laboratory 2. € Laboratory 3. 
Sensitivity was calculated from the number of false negative results within the positive spiked material of sample size 156. 





3.3 Specificity 


The specificity of each laboratory’s testing platform was established using all 115 negative SARS-CoV-2 
samples (Appendix 1). Specificity was determined to be 100%, 99.1% and 97.5% for Laboratory 1, laboratory 
2 and Laboratory 3, respectively (Table 4). Specificity was further evaluated on a subset of 24 samples (9 
VTM samples and 15 samples of 10% saliva in VTM) for test cross-reactivity to other common respiratory 


virus agents (Table 5). These included samples spiked with parvovirus B19, parainfluenza virus 1-3, 
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Real Time PCR platform continued 


coronavirus OC43, coronavirus 229E, rhinovirus, respiratory syncytial virus, adenovirus, influenza virus A and 
B, human metapneumovirus, Coxsackie virus and measles virus. All samples were negative for SARS-CoV-2 
by Laboratory 1. For Laboratory 2, sample 253 (influenza virus A in saliva matrix) and sample 260 
(coronavirus-229E in saliva matrix), were excluded from analysis after returning invalid results (internal 
reference target Ct>32 or not detected). The remaining 22 samples were negative for SARS-CoV-2 by 
Laboratory 2. For Laboratory 3, sample 259 (coronavirus-OC43 in saliva matrix) and sample 260 (coronavirus- 
229E in saliva matrix) were excluded from analysis due to their internal reference target having a Ct>32. The 
remaining 22 samples were negative for SARS-CoV-2 by Laboratory 3. Of note were 4 samples from 
laboratory 3: sample 236 (parainfluenza type-1 in saliva matrix), sample 246 (adenovirus type-5 in saliva 
matrix, sample 181 (rhinovirus in VTM matrix) and sample 191 (influenza virus A in VTM matrix) that 
generated a Ct value above 38 for SARS-CoV-2. These were interpreted as not SARS-CoV-2 positive as per 


manufacturer’s IFU. 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


Table 4. Summary of specificity of each laboratory testing platform for the detection of SARS-CoV-2 (n=115). 


Laboratory 2° 


0/115 1/115 


Laboratory 1° Laboratory 3° 





100% 99.1% 


a Doherty Institute. » Laboratory 2. € Laboratory 3. 
Specificity rate was calculated from the number of negative detections divided by the 115 total true negative sample size. 


Table 5. Results of samples matrices containing non-SARS-CoV-2 respiratory virus agents. 





169 10% saliva sample Parvovirus ND ND ND 
236 10% saliva sample Parainfluenza-1 ND ND ND# 
239 10% saliva sample Parainfluenza-2 ND ND ND 
240 10% saliva sample Parainfluenza-3 ND ND ND 
242 10% saliva sample CoV-O0C43 ND ND ND 
243 10% saliva sample Rhinovirus ND ND ND 
244 10% saliva sample RSV ND ND ND 
246 10% saliva sample Adenovirus type-5 ND ND ND# 
253 10% saliva sample Influenza virus A ND ND* ND 
254 10% saliva sample Influenza virus B ND ND ND 
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255 10% saliva sample Human ND ND ND 
metapneumovirus 

256 10% saliva sample CA16 ND ND ND 
257 10% saliva sample Measles virus ND ND ND 
259 10% saliva sample CoV-O0C43 ND ND ND™ 
260 10% saliva sample CoV-229E ND ND™ ND™ 
131 VTM CoV-229E ND ND ND 
161 VTM Rhinovirus ND ND ND 
175 VTM CoV-229E ND ND ND 
181 VTM Rhinovirus ND ND ND“ 
191 VTM Influenza virus A ND ND ND#* 
225 VTM Influenza virus A ND ND ND 

5 VTM Influenza virus A ND ND ND 

9 VTM CoV-229E ND ND ND 
15 VTM Rhinovirus ND ND ND 


CoV, coronavirus; RSV, respiratory syncytial virus; CA16, Coxsackievirus A16; ND not detected; * SARS-CoV-2 Ct>38 is considered not detected, * Internal 
reference target not detected, ** Internal reference target Ct>32 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


Figure 3: Gaussian distribution of Ct frequency for (a) Laboratory 1 (29.10 +5.98), (b) Laboratory 2 (24.44 +7.05) and 
(c) Laboratory 3 (23.90 +7.63). Values represented are mean +SD. Outliers are represented as solid fill histograms for 
Ct values greater than 38 for BGI workflows and false positives represented as hatched histograms. These outliers were 


excluded in the distribution analysis. 


3.4 Cycle threshold (Ct) value frequency distribution 

During specificity data analysis it was noted that the BGI platform gave Ct values for a number of negative 
samples, both samples containing no target and samples containing other respiratory viruses. In line with 
the manufacturers IFU these did not meet the criteria for positivity, sometimes due to invalid values for the 
internal reference control (232), and more frequently because the IFU states that Ct 238 require retest. This 
was more commonly seen in data from Laboratory 3. Hence the overall specificity values achieved by the 


BGl-users, laboratory 2 & 3 were high after removal of invalid samples from analysis. 


The distribution of Ct values produced by each of the three participating laboratories was calculated and the 
relative frequency displayed as a Gaussian distribution for each data sets (Fig 3 a-c). Outliers are represented 
as solid fill histograms for Ct values greater than 38, which are not considered positive until retesting is 
completed consistent with the IFU. True false positives are represented as hatched histograms at their 
respective Ct values. These latter outliers were excluded in the Gaussian distribution analysis. Differences 
were noted. Frequency data for Laboratory 1 had a mean Ct of 29.10 and a tighter standard deviation of 


+5.98, when compared to both BGI workflows at Laboratory 2 and Laboratory 3. The mean values for 
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Real Time PCR platform continued 


Laboratory 2 and Laboratory 3 were a mean and standard deviation of 24.44 +7.05 and 23.90 +7.63, 


respectively. 


As well as the greater spread of Ct values from BGl-users Laboratory 2 and 3, there was an increased 
frequency of Ct values at 38 and above; most commonly in data from Laboratory 3 (Fig 3 c) where there was 


a small but distinct peak. 


3.5 Comparison of Lot Numbers 


3.5.1 Comparing Sensitivity of Nucleic Acid Extraction kits and RT-PCR kits. 


Nucleic acid was extracted from 45 samples (30 positive spiked material samples and 15 true negative 
samples using the MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit. Two different lot numbers 
(#H0058 and #H0059) at Laboratory 3 were used in this exercise to assess the performance of the BGI’s 
extraction efficiency. The sensitivity was 96.7% and 100% for Lot number #H0058 and #H0059, respectively, 
when the same BGI SLAN 96S RT-PCR kit (lot #6220200318) was used (Table 6). When the same MGIEasy 
Magnetic Beads Virus DNA/RNA Extraction Kit was used the sensitivity was 100% for both BGI SLAN 96S RT- 
PCR kit numbers. The same 45 samples and an additional 45 samples (90 total) were extracted at Laboratory 
1 using the QlAamp 96 DNA QlAcube HT Kit on the QlAcube HT robotic and tested in Laboratory 3 with the 
same BGI RT-PCR kit above. This combination yielded a sensitivity of 95% (Table 7). The same analysis was 
performed at Laboratory 1 across the 3 separate extractions as described above. The in-house RT-PCR 


detection method was consistently used at Laboratory 1. The sensitivity was determined to be 93.3% for BGI 
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extraction (lot #HO058), 90.0% for BGI extraction (lot #HO059) and 88.3% for the Qiagen extraction (Table 6- 
7). 


3.5.2 Comparing Specificity of Nucleic Acid Extraction kits and RT-PCR kits. 


Forty-five samples were extracted for nucleic acid using the MGIEasy Magnetic Beads Virus DNA/RNA 
Extraction Kit. The specificity was 93.8% for Lot number #H0058 and 100% for Lot number #H0059 when the 
same BGI SLAN 96S RT-PCR kit (lot #6220200318) was used (Table 8). When the same MGIEasy Magnetic 
Beads Virus DNA/RNA Extraction Kit was used the specificity was 93.8% and 100% for BGI SLAN 96S RT-PCR 
kit Lot number #6220200319 and #6220200318, respectively. 

The same 45 samples and an additional 45 samples (90 total) were extracted at Laboratory 1 using the 
QlAamp 96 DNA QlAcube HT Kit on the QlAcube HT robotic and tested in Laboratory 3 with the same BGI RT- 
PCR kit above. This combination yielded a specificity of 93.8% (Table 9). The same analysis was performed at 
Laboratory 1 across the 3 separate extractions as described above. The in-house RT-PCR detection method 
was consistently used at Laboratory 1. The specificity was determined to be 100% for all extraction methods 


assessed (Table 8-9). 
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Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
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3.5.3 Comparative Sensitivity and Specificity of Nucleic Acid Extraction Methods on the ABI 7500 


real-time PCR System. 


The sensitivity for Laboratory 1 using the MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit on the 
MGISP-960 extraction robot was 93.3%/90.0% compared to 88.3% for the Qiagen QlAamp 96 DNA QlAcube 
HT Kit with the Qiagen QlAcube HT System. The specificity was 100% irrespective of the nucleic extraction 


kit/platform used. 


3.5.4 Comparative Sensitivity and Specificity of Nucleic Acid Extraction Methods on the BGI SLAN-96S 


real-time PCR system. 


The sensitivity for Laboratory 3 using the MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit on the 
MGISP-960 extraction robot was 96.7%/100%/100% compared to 95.0% for the Qiagen QlAamp 96 DNA 
QlAcube HT Kit with the Qiagen QlAcube HT System. The specificity using the MGIEasy Magnetic Beads Virus 
DNA/RNA Extraction Kit on the MGISP-960 extraction robot was 93.8%/93.8%/100% compared to 93.8% for 
the Qiagen QlAamp 96 DNA QlAcube HT Kit with the Qiagen QlAcube HT System 


Table 6. Detection of positive spiked material with MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit (n=30) 





























Laboratory 1 | Laboratory 1 | Laboratory3 | Laboratory3 | Laboratory 3 
#1 #2 #1 #2 #3 
False 
Negatives 2 : 
Sensitivity (%) 93.3 90.0 96.7 100 100 








True positive rate defined by number of false negatives within the 30 sample size. Laboratory 1 #1 used BGI Extraction Lot# H0058 and 
Laboratory 1 #2 used BGI Extraction Lot# H0059. Laboratory 3 #1 RT-PCR Lot# 6220200318, Laboratory 3 #2 RT-PCR Lot # 6220200319 and 


Laboratory 3 #3 RT-PCR Lot# 6220200318. 
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Table 7. Detection of positive spiked material with Qiagen QlAamp 96 DNA QlAcube HT Kit (n=60) 
Laboratory 1 | Laboratory 3 
False Negatives 7 3 


Sensitivity (%) 88.3 95.0 
True positive rate defined by number of false negatives within the 60 sample size. Laboratory 3 RT-PCR Lot# 6220200318. 


























Table 8. Specificity of detection across laboratories using MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit 
(n=15) 




















Laboratory1 | Laboratory1 | Laboratory3 | Laboratory 3 | Laboratory 3 
#1 #2 #1 #2 #3 
False Positives 0 0 1 1 0 
Specificity (%) 100 100 93.8 93.8 100 

















False positive rate defined by number of false positives within the 15 sample size. Laboratory 1 #1 used BGI Extraction Lot# H0058 and Laboratory 
1 #2 used BGI Extraction Lot# H0059. Laboratory 3 #1 RT-PCR Lot# 6220200318, Laboratory 3 #2 RT-PCR Lot # 6220200319 and Laboratory 3 #3 
RT-PCR Lot# 6220200318. 


Table 9. Specificity of detection across laboratories using Qiagen QlAamp 96 DNA QlAcube HT Kit (n=30) 
Laboratory 1 | Laboratory 3 
False Positives 0 2 


Specificity (%) 100.0 93.8 
False positive rate defined by number of false positives within the 30 sample size. Laboratory 3 RT-PCR Lot# 6220200318. 


























3.6 One-way ANOVA (Dunn's multiple comparisons) 


A One-way ANOVA with Dunn’s multiple comparisons was analysed for all variables, including extraction 
methods and RT-PCR reagents for both Laboratory 1 and Laboratory 3. A standard curve for each specimen 


type of VTM and swabs in UTM was produced via Droplet Digital PCR (ddPCR) using in-house N-gene probe 
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and primers to generate absolute Ct values, as described in the Methods section. The standard curves were 
as follows for VTM and swabs, respectively, where x is copies/mL and y is Ct values: 
y = —3.367 x 10°-°x + 32.08 
y = —2.040 x 1076x + 31.54 


When each variable was compared to the above standard curves and absolute Ct values, there was a 
significant difference in Ct values of BGI extracted material for both Laboratory 3 BGI SLAN 96S RT-PCR kit 
Lot numbers, when swabs in UTM was analysed. For Lot number #6220200319 the calculated p-value 
was 0.0022 and Lot number #6220200318 the calculated p-value of 0.0271 (Table 10). There was no 
significant variation when VTM specimens were used across both extraction methods and RT-PCR 


instruments/reagents for either Laboratory (data not shown). 


Both specimen types showed significant Ct variation within replicates when BGI Virus DNA/RNA Extraction 
Kit was used for nucleic acid extraction at Laboratory 3, and then analysed in Laboratory 1 using the in-house 
RdRP RT-PCR. This significant difference was further pronounced when MGIEasy Magnetic Beads Virus 
DNA/RNA Extraction Kit Lot number #H0059 was used for nucleic acid extraction, and again when swabs in 
UTM was analysed. The VTM specimens also showed Ct variation within replicates for the same workflow as 
described above, however the variation was less pronounced in comparison to the swab specimens in UTM. 


Significant p-values are shown in grey area within Table 10 and Table 11. 


Table 10. One-way ANOVA (Dunn’s multiple comparisons) for swabs in UTM specimens 
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Standard Curve vs. BGI Extraction + Laboratory 3 #2 hie 0.0022 
Standard Curve vs. BGI Extraction + Laboratory 3 #3 z 0.0271 
BGI Extraction: F 0.0058 


Laboratory 1 #1 vs. Laboratory 3 #2 


BGI Extraction: a 0.0028 
Laboratory 1 #2 vs. Laboratory 3 #2 


BGI Extraction: ž 0.0334 
Laboratory 1 #2 vs. Laboratory 3 #3 


Laboratory 1 #1 used BGI Extraction Lot# H0058 and Laboratory 1 #2 used BGI Extraction Lot# H0059. Laboratory 3 #1 RT-PCR Lot# 
6220200318, Laboratory 3 #2 RT-PCR Lot # 6220200319 and Laboratory 3 #3 RT-PCR Lot# 6220200318. 
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Table 11. One-way ANOVA (Dunn’s multiple comparisons) for VTM specimens 





BGI Extraction: 3 0.0419 
Laboratory 1 #2 vs. Laboratory 3 #2 


BGI Extraction: i 0.0299 
Laboratory 1 #2 vs. Laboratory 3 #3 


Laboratory 1 #1 used BGI Extraction Lot# H0058 and Laboratory 1 #2 used BGI Extraction Lot# H0059. Laboratory 3 #1 RT-PCR Lot# 
6220200318, Laboratory 3 #2 RT-PCR Lot # 6220200319 and Laboratory 3 #3 RT-PCR Lot# 6220200318. 


4. Discussion 


A 271 sample validation panel comprising 156 positive and 115 negative samples was used to assess the 
Beijing Genomics institute (BGI) MGISP-960 robotic nucleic extraction instrument, paired with either the 
SLAN 96S real-time PCR analyser (Laboratory 3), or Bioer LineGene 9600 PCR analyser (Laboratory 2), and 
using the MGlEasy Magnetic Beads Virus DNA/RNA Extraction Kit (Ref 1000021043), and BGI Real-time 
fluorescent RT-PCR kit for detecting 2019-nCoV (Ref MFG030010): ‘the BGI platform’. 


The Limit of Detection (LoD) for the BGI platform was found to be a four-fold dilution more sensitive at both 
participating BGl-user laboratories than the Doherty Institute in-house RT-PCR, although one BGl-user 
laboratory did fail to detect one sample replicate at a lower dilution. Had the definition of LoD been rigidly 


applied this would have been scored as a less sensitive LoD. More input RNA is loaded into the BGI RT-PCR 


Page 35 





Doherty o 


Institute THE UNIVERSITY OF Melbourne 


MELBOURNE Hospital 


A joint venture between The University of Melbourne 
and The Royal Melbourne Hospital 





Vv 


Post-market validation of the Beijing Genomics Institute (BGI) SARS-CoV-2 
Real Time PCR platform continued 


assay than the reference method (10yul compared to 1.5ul), and this may contribute significantly to the 


observed relative LoD. 


Operational Sensitivity of the BGI platform was also found to be high. Sensitivity was 98.1% (Laboratory 2) 
and 99.4% (Laboratory 3) for the BGI platform compared to 94.2% for the reference method (Laboratory 1), 
consistent with the observed difference in LoD described above. Overall Operational Specificity was 
determined to be 99.1% (Laboratory 2) and 97.5% (Laboratory 3) compared to 100% for the reference 
method (Laboratory 1). Hence the most sensitive test was the least specific, and vice-versa, although 
differences were small. Specificities of 100% were also demonstrated for both users of the BGI platform in 
testing the sub-group of 24 samples containing other respiratory viruses, but only after spurious reactivity in 
several samples containing other respiratory viruses were excluded from analysis, by either invalid results in 
the test internal control, or recorded Ct values of >38, which according to the IFU may not be reported as 
positive without retesting. Results of retesting were not studied in this exercise. In addition it should be 
noted that such high Operational Specificity is less likely to have been achieved had this part of the study 
been done predominantly using one (#6220200319) of the two lot numbers of RT-PCR reagents which were 


subsequently compared in the sub-study done at Laboratory 3 (see below). 


Examination of the frequency distribution of Ct values recorded by each laboratory revealed differences in 
the spread of values. The BGI platform data shows a greater spread of Ct values, with higher standard 
deviations about the mean (+/-7.05, Laboratory 2 and +/-7.63, Laboratory 3) than the reference method (+/- 


5.98, Laboratory 1). Potentially of greater interest was a small but distinct secondary peak of Ct values 
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beginning at Ct 38, present in data from Laboratory 3, and to a lesser extent from Laboratory 2. Retesting of 
such samples showing Ct >38 is required by the IFU*. Although consistent with the IFU, simply retesting such 
samples using the BGI platform may run the risk that Ct >38 are obtained on re-test, and this might contribute 
to false positive results if reported. However retesting such samples with a different test, which employs a 
different gene target would be recommended practice, both consistent with PHLN guidelines for SARS-COV- 
2 laboratory diagnosis°, and still consistent with the wording of the IFU. Reflexing to a second test potentially 
mitigates possible contribution of high Ct results to falsely positive test results, but it still represents 
potentially onerous levels of retesting if numbers of such high Ct results recorded by Laboratory 3 are 


representative. 


It is not possible to draw conclusions regarding the difference in proportion of samples with Ct>38 observed 
between the two BGl-user laboratories Laboratory 2 and Laboratory 3, since extraction kit and RT-PCR kit lot 


numbers, RT-PCR analyser, and personnel among other variables all differ between the two laboratories. 


Relative sensitivity of lot numbers of the MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit used at 
Laboratory 3 did vary a little between 96.7% (#H0058) and 100% (#HO059) when compared using the same 
Lot number (#6220200318) of the BGI Real-time fluorescent RT-PCR kit for detecting 2019-nCoV. However 
the opposite trend in sensitivity was observed when these same two MGlEasy Magnetic Beads Virus 
DNA/RNA Extraction Kit Lot numbers were used with the reference RT-PCR method in Laboratory 1 (#HO058, 


93.3%, and #H0059, 90%). Hence this variation is of doubtful significance. Use of the reference QlAamp 
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QlAcube HT kit with BGI Real-time fluorescent RT-PCR kit (Lot #6220200318) for detecting 2019-nCoV yielded 


a sensitivity of 95% by way of comparison. 


Specificity of lot numbers #H0058 and #H0059 of the MGIEasy Magnnetic Beads Virus DNA/RNA Extraction 
Kit used at Laboratory 3 did not vary; both being 93.8% when the same Lot number (#6220200318) of the 
BGI Real-time fluorescent RT-PCR kit for detecting 2019-nCoV was used. Use of the reference QlAamp 
QlAcube HT kit with the same Lot number of the BGI Real-time fluorescent RT-PCR kit (Lot #6220200318) for 
detecting 2019-nCoV yielded the same sensitivity of 93.8%. However specificity was found to be 100% using 
either of the two MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit Lot numbers #HO058 and #H0059 
together with the reference RT-PCR method in Laboratory 1. One hundred percent versus 93.8% was a bigger 
difference in specificity within this sub-study than was observed when the entire BGI-platform was compared 
to the reference method (see above) and high values of 99.1% and 97.5% were obtained for Operational 


Specificity by the BGl-users. 


Sensitivity of the BGI Real-time fluorescent RT-PCR kit for detecting 2019-nCoV did not vary between lot 
numbers #6220200318 and #6220200319; both 100%, when the same lot number of the MGIEasy Magnetic 
Beads Virus DNA/RNA Extraction Kit was used (Lot #HO059). However specificity of different Lot numbers of 
the BGI Real-time fluorescent RT-PCR kit for detecting 2019-nCoV did vary considerably between 93.8% 
(#6220200319), and 100% (#6220200318) when the same lot number of the MGIEasy Magnetic Beads Virus 
DNA/RNA Extraction Kit was used (Lot #H0059). This may have contributed to the observed difference is 


specificity between the substudy described here, and the high specificities achieved with the whole 271 
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sample panel, mostly done using lot numbers other than #6220200319 (i.e. predominantly with 


#6220200318). 


The use of droplet digital PCR (ddPCR) provided objective quantification of selected samples from within the 
panel: 4-fold dilution series of cell culture-grown SARS-Cov-2 spiked into VTM, swabs, and 10% saliva diluted 
in VTM. This allowed statistical comparison of the degree of precision achieved in testing sample replicates 
using one-way Analysis of Variance (ANOVA) statistics. Overall the observed precision between methods was 
similar. However, statistically significant differences in precision were seen between reference ddPCR Ct 
values and BGI platform Ct values for swabs diluted in UTM, with both lot numbers of the BGI Real-time 
fluorescent RT-PCR kit for detecting 2019-nCoV Lot numbers (#6220200319, p=0.0022 and #6220200318, 
p=0.0027), using the same lot number of the MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit (Lot 
#H0059). This observation did not extend to other sample types. Both swabs in UTM, and VTM samples 
demonstrated significant variation within replicates when the MGIEasy Magnetic Beads Virus DNA/RNA 
Extraction Kit was used at Laboratory 3, and RT-PCR was subsequently done by the reference method at 


Laboratory 1. This was more pronounced with Lot #H0059. 


In summary the BGI platform was shown to have a LoD, and Operational Sensitivity and Specificity that was 
comparable to the reference method using the 271 sample panel of simulated clinical samples of cell-culture 
grown SARS-CoV-2, and other respiratory viruses spiked into sample matrices. This was fairly consistently 
achieved in two BGl-user Laboratories. A potential tendency of the BGI RT-PCR assay to generate high Ct 


values of 38 and above for negative samples is mitigated with a threshold for retesting at Ct >38 specified in 
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the IFU. Use of reflex testing to a second assay using a different gene target for this retesting would be 


consistent with Australian PHLN guidelines, and could mitigate any inherent risk of false positive reactivity. 


One lot number of BGI Real-time fluorescent RT-PCR reagents gave noticeably lower specificity than another, 
and varied considerably from the overall Operational Specificity achieved when testing the entire 271 sample 
panel. Otherwise different lot numbers of MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit and BGI 
Real-time fluorescent RT-PCR reagents gave similar performance. Some statistically significant differences in 
precision between sample replicates were observed, but these were more evident with swabs diluted in UTM 
than with other samples, and with one batch of MGIEasy Magnetic Beads Virus DNA/RNA Extraction Kit 
reagents. Over all specimen types the degree of inter-sample precision achieved was acceptable in a 


qualitative assay. 


Hence no single issue has been identified to suggest that the BGI platform is not fit for purpose in 
Australian laboratories. Best practice in executing confirmatory testing of samples producing Ct values 2 38 
will be important in use. Similarly, close monitoring of consistency in performance of Lot numbers, 


especially of RT-PCR reagents, will be important to ensure optimal performance in clinical use 
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Appendix A. Line listing of Panel Samples and SARS-CoV-2 RT-PCR Results 






Sample 


Specimen 


Positive spiked 


SARS-CoV-2 RT-PCR result” 






























































number matrix material Laboratory 1 Laboratory2 | Laboratory 3 
1 VTM SARS-CoV-2 24.1 19.9 19.3 
2 VTM SARS-CoV-2 26.1 22.4 21.5 
3 VTM SARS-CoV-2 28.6 24.3 23.4 
4 VTM None ND ND ND 
5 VTM Influenza virus A ND ND ND 
6 VTM SARS-CoV-2 30.6 26.5 25.8 
7 VTM None ND ND ND 
8 VTM SARS-CoV-2 32.5 28.7 27.3 
9 VTM CoV-229E ND ND ND 
10 VTM SARS-CoV-2 31.0 26.6 25.5 
11 VTM None ND ND 39.0 
12 VTM SARS-CoV-2 24.0 20.4 19.7 
13 VTM SARS-CoV-2 26.1 22.3 21.5 
14 VTM SARS-CoV-2 28.8 24.4 23.6 
15 VTM Rhinovirus ND ND ND 
16 VTM SARS-CoV-2 33.0 28.6 27.7 
17 VTM SARS-CoV-2 33.0 31.6 30.0 
18 VTM 1/1,024,000 33.7 31.3 30.2 

101 Saliva None ND ND ND 
102 Saliva None ND ND 37.1 
103 Saliva None ND ND ND 
104 Saliva SARS-CoV-2 18.5 15.6 14.9 
105 Saliva SARS-CoV-2 18.9 15.7 15.2 
106 Saliva None ND ND ND 
107 Saliva None ND ND ND 
108 Saliva SARS-CoV-2 19.8 16.4 15.9 
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109 Saliva None ND ND ND 
110 Saliva None ND ND ND 
111 Saliva None ND ND ND 
112 Saliva None ND ND 39.1 
113 Saliva SARS-CoV-2 18.1 15.5 15.7 
114 Saliva None ND ND ND 
115 Saliva None ND ND ND 
116 Saliva None ND ND ND 
117 Saliva SARS-CoV-2 20.5 16.3 15.9 
118 Saliva None ND ND ND 
119 Saliva None ND ND 39.9 
120 Saliva None ND ND ND 
121 VTM SARS-CoV-2 23.9 21.5 21.2 
122 VTM SARS-CoV-2 31.0 28.3 27.9 
123 VTM SARS-CoV-2 31.5 28.5 28.1 
124 VTM SARS-CoV-2 34.1 31.9 30.8 
125 VTM SARS-CoV-2 40.2 36.8 34.9 
126 VTM SARS-CoV-2 24.1 21.4 21.1 
127 VTM None ND ND 38.8 
128 Saliva None ND ND ND 
129 VTM SARS-CoV-2 27.2 24.9 24.6 
130 VTM SARS-CoV-2 27.2 24.8 24.6 
131 VTM CoV-229E ND ND ND 
132 VTM SARS-CoV-2 28.6 25.7 25.0 
133 VTM None ND ND ND 
134 Saliva None ND ND ND 
135 VTM SARS-CoV-2 24.4 21.0 21.2 
136 VTM None ND ND ND 
137 VTM SARS-CoV-2 34.8 32.1 31.6 
138 VTM SARS-CoV-2 26.1 26.2 23.1 
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139 VTM SARS-CoV-2 33.0 29.4 29.3 
140 VTM SARS-CoV-2 28.8 25.6 25.1 
141 Saliva None ND ND ND 
142 Saliva None ND ND ND 
143 Saliva None ND ND ND 
144 Saliva None ND 39.2 ND 
145 Saliva None ND ND ND 
146 Saliva None ND ND ND 
147 Saliva None ND ND ND 
148 Saliva None ND ND 39.2 
149 Saliva SARS-CoV-2 19.2 15.4 15.1 
150 Saliva None ND ND ND 
151 Saliva None ND ND ND 
152 Saliva None ND ND 39.7 
153 Saliva None ND ND ND 
154 Saliva None ND ND ND 
155 Saliva SARS-CoV-2 20.2 16.6 16.2 
156 Saliva None ND ND ND 
157 Saliva None ND ND ND 
158 Saliva None ND ND ND 
159 Saliva None ND ND ND 
160 Saliva None ND ND 38.6 
161 VTM Rhinovirus ND ND ND 
162 VTM SARS-CoV-2 37.0 28.6 28.0 
163 Saliva SARS-CoV-2 20.6 16.5 15.8 
164 Saliva None ND ND ND 
165 Saliva SARS-CoV-2 19.7 15.6 15.2 
166 VTM SARS-CoV-2 24.7 21.7 20.8 
167 VTM SARS-CoV-2 37.6 38.4 35.3 
168 VTM SARS-CoV-2 27.6 24.6 24.2 
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169 Saliva Parvovirus ND ND ND 
170 Saliva None ND ND ND 
171 Saliva SARS-CoV-2 19.7 15.6 14.8 
172 VTM SARS-CoV-2 31.2 27.8 27.1 
173 VTM SARS-CoV-2 24.3 21.6 20.9 
174 VTM SARS-CoV-2 26.4 23.6 23.1 
175 VTM CoV-229E ND ND ND 
176 Saliva None ND ND ND 
177 Saliva None ND ND 40.0 
178 VTM SARS-CoV-2 31.3 28.5 28.0 
179 VTM SARS-CoV-2 35.5 32.0 31.1 
180 VTM SARS-CoV-2 ND 35.1 34.9 
181 VTM Rhinovirus ND ND 39.3 
182 VTM SARS-CoV-2 33.0 29.7 29.4 
183 VTM SARS-CoV-2 28.3 24.9 24.3 
184 VTM SARS-CoV-2 28.0 24.8 24.3 
185 VTM SARS-CoV-2 30.9 27.7 27.3 
186 VTM SARS-CoV-2 26.5 23.5 23.1 
187 VTM SARS-CoV-2 31.0 27.9 27.3 
188 VTM SARS-CoV-2 30.6 27.9 27.1 
189 VTM SARS-CoV-2 29.0 25.5 25.0 
190 Saliva SARS-CoV-2 19.8 16.7 15.7 
191 VTM Influenza virus A ND ND 39.7 
192 Saliva None ND ND ND 
193 VTM SARS-CoV-2 31.2 28.6 27.5 
194 VTM SARS-CoV-2 32.7 29.8 28.9 
195 VTM None ND ND ND 
196 VTM SARS-CoV-2 24.4 21.4 20.5 
197 VTM SARS-CoV-2 35.1 31.8 31.2 
198 VTM SARS-CoV-2 35.2 31.9 31.2 
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199 Saliva SARS-CoV-2 20.4 16.8 16.2 
200 VTM None ND ND ND 
201 VTM SARS-CoV-2 39.6 ND 34.8 
202 Saliva SARS-CoV-2 21.8 17.2 16.3 
203 Saliva None ND 32.8 ND 
204 Saliva None ND ND ND 
205 Saliva SARS-CoV-2 18.8 15.9 14.8 
206 Saliva None ND ND ND 
207 Saliva SARS-CoV-2 25.1 16.8 16.2 
208 Saliva None ND ND ND 
209 Saliva SARS-CoV-2 20.9 16.4 15.8 
210 Saliva SARS-CoV-2 20.4 16.4 15.7 
211 Saliva None ND ND ND 
212 VTM SARS-CoV-2 24.3 21.3 20.6 
213 VTM SARS-CoV-2 ND 35.4 33.8 
214 VTM SARS-CoV-2 32.5 29.4 28.9 
215 Saliva SARS-CoV-2 20.3 16.8 16.3 
216 Saliva None ND ND ND 
217 VTM SARS-CoV-2 34.7 31.7 31.2 
218 Saliva SARS-CoV-2 21.0 16.6 16.2 
219 VTM SARS-CoV-2 31.8 28.5 27.3 
220 Saliva None ND ND ND 
221 VTM SARS-CoV-2 30.8 28.0 27.4 
222 Saliva SARS-CoV-2 20.1 17.1 16.3 
223 VTM SARS-CoV-2 27.4 24.5 24.1 
224 VTM SARS-CoV-2 26.1 23.4 22.8 
225 VTM Influenza virus A ND ND ND 
226 VTM None ND ND ND 
227 VTM SARS-CoV-2 29.4 25.5 24.9 
228 Saliva SARS-CoV-2 21.1 15.5 14.7 
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229 Saliva None ND ND ND 
230 VTM SARS-CoV-2 35.1 32.6 31.2 
231 VTM SARS-CoV-2 39.7 35.8 35.2 
232 VTM SARS-CoV-2 38.4 36.0 34.2 
233 VTM SARS-CoV-2 25.3 20.9 20.4 
234 Saliva SARS-CoV-2 21.2 16.9 16.2 
235 VTM SARS-CoV-2 31.7 28.4 27.4 
236 Saliva Parainfluenza-1 ND ND 39.2 
237 VTM SARS-CoV-2 24.4 20.9 20.5 
238 VTM SARS-CoV-2 35.3 31.7 31.4 
239 Saliva Parainfluenza-2 ND ND ND 
240 Saliva Parainfluenza-3 ND ND ND 
241 Saliva SARS-CoV-2 21.8 16.8 16.4 
242 Saliva CoV-0C43 ND ND ND 
243 Saliva Rhinovirus ND ND ND 
244 Saliva RSV ND ND ND 
245 VTM SARS-CoV-2 28.4 24.6 24.1 
246 Saliva Adenovirus type-5 ND ND 39.4 
247 Saliva SARS-CoV-2 20.0 16.5 15.8 
248 Saliva None ND ND 37.7 
249 VTM SARS-CoV-2 26.0 21.5 20.6 
250 VTM SARS-CoV-2 28.0 24.7 24.0 
251 VTM SARS-CoV-2 44.9 34.3 34.5 
252 VTM SARS-CoV-2 26.4 21.0 20.4 
253 Saliva Influenza virus A ND ND ND 
254 Saliva Influenza virus B ND ND ND 
255 Saliva hMPV ND ND ND 
256 Saliva CA16 ND ND ND 
257 Saliva Measles virus ND ND ND 
258 VTM SARS-CoV-2 24.0 21.3 20.7 
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259 Saliva CoV-OC43 ND ND ND 
260 Saliva CoV-229E ND ND ND 
261 VTM 1/64,000 28.6 26.5 26.1 
262 VTM 1/4,000 24.4 21.8 21.5 
263 VTM 1/65,536,000 ND 36.3 35.7 
264 Swab SARS-CoV-2 28.1 20.3 19.7 
265 Swab SARS-CoV-2 33.5 22.4 26.0 
266 Swab None ND ND ND 
267 Swab None ND ND 38.0 
268 VTM 1/16,000 27.6 24.6 24.0 
269 VTM SARS-CoV-2 33.0 30.6 30.1 
270 Swab SARS-CoV-2 28.2 24.2 20.7 
271 Swab None ND ND ND 
272 VTM 1/16,384,000 38.3 34.5 34.9 
273 VTM 1/262,144,000 ND ND 37.4 
274 Swab SARS-CoV-2 31.0 28.8 24.1 
275 Swab SARS-CoV-2 34.2 26.8 28.0 
276 VTM 1/256,000 32.2 28.7 28.2 
277 Swab SARS-CoV-2 30.5 28.7 23.8 
278 Swab None ND ND ND 
279 Swab None ND ND ND 
280 VTM 1/4,096,000 36.1 32.7 32.6 
281 VTM 1/262,144,000 ND 38.7 ND 
282 Swab SARS-CoV-2 30.7 28.2 23.5 
283 Swab None ND ND ND 
284 Swab None ND ND ND 
285 Swab None ND ND ND 
286 VTM 1/4,000 24.2 21.7 21.4 
287 Swab SARS-CoV-2 35.9 26.6 28.2 
288 VTM 1/4,000 25.1 21.9 21.3 
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289 Swab None ND ND ND 
290 Swab None ND ND ND 
291 Swab None ND ND ND 
292 Swab None ND ND ND 
293 VTM 1/256,000 32.4 28.5 27.6 
294 Swab SARS-CoV-2 33.0 22.4 25.8 
295 Swab None ND ND ND 
296 Swab None ND ND ND 
297 Swab None ND ND ND 
298 Swab None ND ND ND 
299 VTM 1/1,024,000 33.4 30.5 29.8 
300 VTM 1/4,096,000 36.9 32.6 32.6 
301 VTM 1/16,384,000 38.9 ND 34.2 
302 Swab SARS-CoV-2 2575 20.4 20.1 
303 Swab SARS-CoV-2 32.0 28.5 23.4 
304 Swab SARS-CoV-2 34.8 22.0 26.2 
305 Swab SARS-CoV-2 33.5 26.7 27.8 
306 Swab None ND ND ND 
307 VTM 1/4,000 26.1 21.7 21.2 
308 Swab None ND ND ND 
309 Swab None ND ND ND 
310 VTM 1/16,000 29.6 24.4 24.0 
311 VTM 1/1,024,000 34.1 30.5 30.2 
312 Swab SARS-CoV-2 28.3 20.3 19.9 
313 Swab SARS-CoV-2 31.1 20.8 21.8 
314 Swab SARS-CoV-2 29.0 21.6 21.7 
315 Swab None ND ND ND 
316 Swab None ND ND ND 
317 VTM 1/64,000 29.9 26.4 26.1 
318 Swab SARS-CoV-2 ao 22.5 25.6 
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319 VTM 1/65,536,000 ND 36.4 36.5 
320 Swab SARS-CoV-2 29.3 24 21.2 
321 VTM 1/4,096,000 36.4 32.7 32.0 
322 Swab None ND ND ND 
323 Swab SARS-CoV-2 28.1 24.4 20.8 
324 Swab None ND ND ND 
325 Swab None ND ND ND 
326 Swab None ND ND ND 
327 Swab None ND ND ND 
328 VTM 1/64,000 30.4 26.7 25.7 
329 VTM 1/256,000 32.5 28.4 28.1 
330 VTM 1/16,384,000 36.3 33.6 34.1 
331 Swab SARS-CoV-2 27.5 20.9 21.5 
332 Swab None ND ND ND 
333 Swab None ND ND ND 
334 Swab None ND ND ND 
335 VTM 1/262,144,000 40.2 37.9 36.7 
336 Swab None ND ND ND 
337 Swab None ND ND ND 
338 Swab None ND ND ND 
339 VTM 1/1,024,000 34.0 30.6 29.3 
340 Swab None ND ND ND 
341 VTM 1/65,536,000 ND 36.8 34.8 
342 VTM 1/262,144,000 ND 37.4 36.1 
343 Swab SARS-CoV-2 28.2 24.5 20.2 
344 VTM 1/16,000 28.2 24.5 23.4 
345 VTM 1/16,000 28.1 24.3 23.6 
346 VTM 1/4,096,000 36.1 32.7 31.6 
347 Swab SARS-CoV-2 29.6 21.2 22.0 
348 Swab SARS-CoV-2 38.5 26.4 27.2 
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349 VTM 1/256,000 31.7 28.4 27.7 
350 VTM 1/65,536,000 ND 36.4 35.3 
351 Swab SARS-CoV-2 28.4 20.2 19.3 
352 VTM 1/64,000 30.0 26.2 25.2 
353 VTM 1/16,384,000 37.8 34.9 33.9 
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